Gradient Coding
نویسندگان
چکیده
We propose a novel coding theoretic framework for mitigating stragglers in distributed learning. We show how carefully replicating data blocks and coding across gradients can provide tolerance to failures and stragglers for synchronous Gradient Descent. We implement our scheme in MPI and show how we compare against baseline architectures in running time and generalization error.
منابع مشابه
Wideband, wide-angle coding phase gradient metasurfaces based on Pancharatnam-Berry phase
A new concept of the coding phase gradient metasurface (CPGM) is proposed, which is constructed using the phase gradient metasurface as the coding elements. Different from the previous coding metasurface (CM), both the coding sequences and gradient phases in the coding elements are designed to manipulate the electromagnetic (EM) wave for the CPGMs, and thus the manipulation will be more flexibl...
متن کاملGradient Coding: Avoiding Stragglers in Distributed Learning
We propose a novel coding theoretic framework for mitigating stragglers in distributed learning. We show how carefully replicating data blocks and coding across gradients can provide tolerance to failures and stragglers for synchronous Gradient Descent. We implement our schemes in python (using MPI) to run on Amazon EC2, and show how we compare against baseline approaches in running time and ge...
متن کاملImage Characteristic Based Rate Control Algorithm for HEVC
Rate control plays an important role in high quality video coding. In this paper, a rate control algorithm based on video characteristics is proposed. Firstly, we put forward a novel bit allocation algorithm for intra frame by analyzing the relationship among the image characteristic, bit per pixel and quantization step. The image gradient is used as image characteristic. In frame layer, differ...
متن کاملGOYAL & VETTERLI : ADAPTIVE TRANSFORM CODING USING LMS - LIKE . . . 1 Adaptive Transform Coding UsingLMS - like Principal Component
A new set of algorithms for transform adaptation in adaptive transform coding is presented. These algorithms are inspired by standard techniques in adaptive nite impulse response (FIR) Wiener ltering and demonstrate that similar algorithms with simple updates exist for tracking principal components (eigenvectors of a correlation matrix). For coding an N-dimensional source, the transform adaptat...
متن کاملApproximate Gradient Coding via Sparse Random Graphs
Distributed algorithms are often beset by the straggler effect, where the slowest compute nodes in the system dictate the overall running time. Coding–theoretic techniques have been recently proposed to mitigate stragglers via algorithmic redundancy. Prior work in coded computation and gradient coding has mainly focused on exact recovery of the desired output. However, slightly inexact solution...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
- CoRR
دوره abs/1612.03301 شماره
صفحات -
تاریخ انتشار 2016